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ABSTRACT 

There have been many articles written about the variety of ways faculty and students define eportfolios, collect artifacts, 
write reflections, and use finished eportfolios. However, little has been written about the dynamic process of adopting 
electronic portfolios in schools of education. This study employs cross-case analysis to investigate implementation of 
electronic portfolio initiatives in seven schools of education. Case analysis shows the most commonly identified purpose 
for initiating an eporffolio system to be program evaluation for national accreditation. With program evaluation as the 
eventual goal of eporffolio development, at least eight design steps of an electronic portfolio process were found to be 
necessary for successful implementation. These steps are discussed and various methods of implementation are 
described. Case analysis results further suggest that the final step involving data gathering, aggregation and analysis has 
been most difficult for these schools to implement. None of the schools represented in this study have yet made use the 
data collected from eportf olios to make data driven decisions about their programs of study. Suggestions are made for 
further study into the use of eporffolio data for program evaluation, improvement and ongoing development. 

Keywords: electronic portfolio, ePortf olio, assessment, program evaluation, case study, assessment systems, assessment 
technology. 


INTRODUCTION 

Electronic portfolios have become a part of many 
programs in colleges of education. Electronic portfolios 
are typically based upon national or state professional 
standards, and are used as a way for students to 
demonstrate that they have met these standards through 
their program of study. Because portfolios are repositories 
of student work, they are often called upon to serve 
program evaluation purposes. However, institutions of 
higher education are finds that for electronic portfolio 
systems requires to be engaged in complex planning and 
implementation processes to be used successfully for 
program evaluation. This study attempts to identify the 
steps in successful implementation of electronic 
portfolios within several schools of education. By learning 
from past experiences with electronic portfolios and by 
studying the steps other institutions of higher education 


have taken toward implementation, it is more likely that 
working electronic portfolio systems can be developed. 
Portfolios have many important purposes within a college 
of education or program of study. Richard Stiggins (2005, 
p. 320) describes a student portfolio as "a collection of 
student work assembled to provide a representation of 
student achievement that is a function of the context, the 
purpose for assessing and communicating, and the 
learning targets" (goals and objectives) that are being 
assessed. Barrett (2005) describes portfolios as doing 
more than showcasing student accomplishments. They 
provide a rich picture of student work that documents 
growth over time. Mary Diez (cited in Gibson & Barrett, 
2002) describes the portfolio as the mirror, the map, and 
the sonnet of a student's life. The benefits to students can 
include opportunities for increased learning effectiveness 
as well as opportunities to model professionalism. 
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Portfolios are also used as an instructional method to 
support reflections that are designed to help students 
understand their own learning. NBPT standards describe 
three levels of writing competency important for 
practicing teachers including: description, analysis and 
reflection. Assessment of student writing at all three levels 
is facilitated through the use of portfolios. Further, 
portfolios have the potential to make learners more 
personally responsible for the achievement of 
educational goals and professional standards. Teacher 
candidates trained and experienced in the use of 
portfolios are thought to have a competitive advantage. 
They can use their portfolio to demonstrate employment- 
related skills, as well as disciplinary expertise, and are 
better prepared to seek national certification (NBPTS). 
Portfolios promise a viable alternative to current, high- 
stakes testing, which focuses education on test-taking 
rather than teaching and learning. Through the use of 
portfolios, colleges of education give training for students 
to demonstrate their own learning and 
accomplishments, and are using this documentation for 
program evaluation purposes. 

Implementing electronic or web-based portfolios is 
complex and difficult for any institution. Organizational 
structures and processes must be in place to provide 
portfolio systems that can manage input from multiple 
users as well as huge amount of data. Love, et. al. (2004) 
noted that authentic data must come from the 
collaborative work of students, teachers, and 
administrators. The best systems have methods for 
converting data into information for purposes of analysis, 
and can provide formative as well as summative 
feedback. Jafari (2004) discusses the "challenges in order 
to turn an electronic portfolio concept into a working 
system" (p. 38) and defines several steps to develop the 
system: conceptual design (functional and technical 
requirements), software design (human and computer 
aspects), and implementation plan (business plan, daily 
operation, and software upgrades). 

This study examines the process of electronic portfolio 
implementation in seven different colleges of education 
in order to identify steps in the implementation process. 


Fullan (2001) describes change as a "detailed and 
snarled process" (p. 50). Fullan further notes that 
feedback about a change process gained through 
implementation in one phase can alter decisions made 
at earlier stages in the process. While, understanding that 
change is never a linear process and that no two 
institutions will implement a change process in exactly the 
same manner, determining how schools of education 
have implemented electronic portfolio systems - from 
their introduction through their use for program evaluation 
purposes - informs those who are considering 
development of ePortfolio systems. 

Methods 

This study employs case methodology with cross-case 
analysis to investigate implementation of electronic 
portfolios in seven teacher education programs. The 
study was initiated as the result of interactions between 
faculty members from seven different colleges of 
education across the United States. Information was 
exchanged over the 2004 - 2005 school year via a 
professional listserv on the topic of electronic portfolios. 
Online conversations about electronic portfolio 
implementation produced a proposal for a panel 
discussion, which was held at the 2005 NECC conference 
in Philadelphia. Each member of the panel reviewed the 
literature and wrote a full paper on the electronic portfolio 
implementation process in their respective schools. Each 
paper followed a similar format, based on a template 
agreed between panel members. These papers were 
shared on a website, and discussed online as well as in 
telephone conference calls. This study is a review and 
synthesis of those formal papers written by each member 
of that panel in June 2005. The seven sites examined 
included: Black Hills State University (BHSU), Lesley 
University (Lesley), National College of Education, 
National-Louis University (NLU), University of Maryland 
College Park (UMD), Valdosta State University (Valdosta), 
University of Vermont (UV), and Wake Forest University 
(WFU). Drafts of this paper were reviewed by all 
participants. 

Findings 

Each of the case studies was analyzed and cross-case 
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comparisons were completed. Case analysis was 
organized by the following themes: 1. rationale for 
electronic portfolios: 2. The methods used to establish 
electronic portfolio systems: 3. the components 
necessary for a functional electronic portfolio system 
adequate for all purposes defined by each school: 4. 
changes that were necessary during the implementation 
process and the reasons for those changes: and 5. 
internal and external environmental factors that 
presented challenges. A detailed analysis of each case 
study was also conducted to determine the types and 
nature of the technologies used to support electronic 
portfolio systems. 

What were the purposes defined for portfolios? 

Each of the seven schools in this case study had a paper- 
based portfolio process in place before they moved 
toward adoption of electronic portfolios. Paper based 
portfolios served a range of purposes. Most common 
among them in schools was the need to provide 
evidence of student achievement of professional 
standards: so that students could gain licensure and/or 
employment. Evidence of student achievement was also 
needed by each institution for program evaluation and 
accreditation. 

Following is a list of the purposes for paper portfolios as 
identified by the seven institutions of higher education 
taken for research: 

• Storage space for student work samples, artifacts, 
multimedia files, digital video files, etc. 

• Student demonstration of knowledge, skills and 
understanding gained through their program of 
study. 

• Assessment of student learning and achievement in a 
course and/or throughout a program of study 

• Documentation showing that students have met 
state, national and/or professional standards. 

• Documentation that students have met requirements 
for professional licensure. 

• Documentation of professional growth overtime. 

• Performance records helping students make 


connections between theory and practice 

• Performance records promoting collaboration 
among students and their classmates, course 
instructors, and seminar leaders. 

• Support for reflections on learning and teaching, on 
curriculum, pedagogy, and learning theory, and for 
student self-evaluation. 

• Reflections to reinforce practice and professional 
growth overtime. 

• Evidence used as the basis for feedback to students 
about course/program outcomes. 

• To showcase skills for employment and other 
professional purposes. 

For Program evaluation. 

• Evidence assuring alignment between courses and 
field experiences. 

• Evidence used to ensure that each course 
assignment is unique to that course across a program 
of study. 

• Evidence demonstrating that the program of study 
has helped to students meet goals and professional 
standards. 

• Evidence and data for NCATE/ TEAC accreditation. 
Data for national, state, and university reports. 

• Several of the schools were responding to state 
portfolio requirements for pre-service and/or 
practicing teachers. These state accredited teacher 
preparation programs had responsibility for 
recommending students for initial licensure based 
upon a portfolio demonstrating student fulfillment of 
state standards. 

What factors contributed to the shift toward electronic 
portfolios? 

Program review by the National Council for Accreditation 
of Teacher Education (NCATE) was the driving force 
behind vigorous exploration and eventual adoption of 
electronic portfolio system in several of the schools. In 
each of the seven schools, electronic portfolios slowly 
began to replace other types of portfolios. In most of the 
schools, the choice of creating an electronic portfolio 
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was initially optional and the decision to use them left to 
faculty and/or the students, on a course-by-course or 
program basis. Gradual acceptance for the use of 
technology, accompanied by the need for efficient 
methods of documenting student achievement 
eventually resulted in wider electronic portfolio initiatives. 
Two case examples are provided below: 

WFU: Accreditation requirements have also influenced 
the use of electronic portfolios by increasing faculty 
appreciation for using technology to easily archive 
student performance. External expectations influenced 
the faculty to embrace more efficient methods for 
capturing, storing, and presenting documentation of 
candidate performance. The use of electronic portfolios 
has also been in response to faculty acceptance and 
appreciation of students' ability to use digital tools, 
access to new technologies (both software and 
hardware), and program specific needs/desires for 
demonstration of candidate knowledge, skills, and 
dispositions. Evolving technologies most certainly permit 
faculty to design portfolio expectations that relate to what 
is known about quality teaching and technology 
integration. Aim of selecting electronic portfolio options is 
for supporting the needs of the students and the teacher 
education curriculum. It is important for students to 
develop technology skills. The move to electronic 
portfolios enhances the development of their technology 
skills for communication and demonstration of abilities, 
dispositions, and practice. 

UV: State accredited teacher preparation programs were 
given responsibility for recommending students for initial 
licensure based upon a portfolio that demonstrated their 
fulfillment of state standards. The electronic portfolio is 
neither systematic nor non systematic, and is still evolving. 
However, faculty experiencing students' electronic 
portfolios have noted easier navigation and substantially 
lighter portfolios to transport. In addition, faculty notes that 
electronic portfolios can be easily duplicated, stored, 
shared, and used as models for other students. 

What roles the other factors played on the adoption of 
electronic portfolios? 

In each of the schools, the need for evidence of student 


mastery of program goals and applicable standards, as 
well as the need for aggregated assessment data for 
accreditation and program evaluation purposes 
eventually became imperative, and it was found that 
electronic portfolios functioned better for these purposes 
than did paper portfolios. Case examples are provided 
below: 

UMD: Gearing up for the National Council for 
Accreditation of Teacher Education (NCATE) visit (along 
with their new requirements) seemed to be the driving 
force to have the College explore more vigorously 
adapting an e-portfolio system. 

BHSU: Changes in NCATE accreditation requirements are 
pushing the university toward use of electronic portfolios. 
Changes in state and national standards make it 
importantto consider use of electronic portfolios. 

NLU: As NCATE moved toward standards-based 
assessment and the state of Illinois established standards 
for certification, the portfolio became an end-of- 
program capstone- benchmark assessment. With NCATE, 
ISTE, and state reviews of the program, the portfolio 
requirement became a standards-based assessment, 
with less focus on documenting candidate growth during 
their participation in the program. 

In all seven case studies, there was enough top-down and 
bottom-up leadership to result in an administrative 
mandate for the use of electronic portfolios. Further, as 
mandates moved forward, other faculty and staff 
became involved. The vision was either sold to others on 
campus or was rapidly adopted by the community, or a 
combination of both. 

WFU: Electronic portfolios were adopted by faculty in the 
Department of Education in response to ISTE NETS*T in 
1999 and are an integral part of the curriculum. 

Valdosta: Dean of the College of Education (COE) 
supported electronic portfolios, and faculty 
representatives from every department in the College of 
Education, were invited to work on the eportfolio 
committee. 

Lesley: A state accreditation agency (NEASC) made 
recommendations for more effective program 
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evaluation. In response, a Director of Assessment and 
Program Evaluation was hired and a campus-wide 
electronic portfolio committee was created. Though an 
electronic portfolio adoption process has not yet been 
fully initiated, several programs of study have begun pilot 
projects. 

What are the steps involved in the implementation of an 
electronic portfolio system sufficient to address all 
important assessment purposes? 

Cross case comparisons identified eight design steps in 
an electronic portfolio system that each school had put in 
place to ensure that electronic portfolios could be used 
for all identified purposes, including program evaluation. 
Those design steps are listed below: 

Design Step 1: Defining the purposes forePortfolios 
Design Step 2: Defining the goals and standards to be 
assessed via ePortfolios. Multiple sets of standards were 
typically identified by the schools of education in this 
study. 

Design Step 3: Curriculum mapping - matching goals 
and standards to the requirements of a program of study, 
by course, by licensure area, or for the whole program of 
study. 

DesignStep4: Determination of the ePortfolio 
technologies to be used. 

Design Step 5: Descriptions of portfolio requirements - 
instructions providing information about the role of 
students in the portfolio process, as well as the types and 
nature of artifacts to be included. 

DesignStep6: Assessment systems for portfolio 
artifacts. Assessment methods need to be defined for 
evaluation of student work and for determining whether 
students have met goals, standards and/or licensing 
requirements. 

Design Step 7: Program evaluation systems for data 
gathering, data aggregation, and analysis. 

Design Step 8: Using data to provide feedback for 
improving teaching, learning and program design. 

The methods each school used to implement these 
components were examined and the technologies used 


by each school were identified. Below is a discussion of 
how the various schools made these Design steps 
operational: 

Design Steps 2 and 3 - Standards identification and 
curriculum mapping. 

Program evaluation for NCATE, TEAC, or regional 
accreditation agencies became the most important 
purpose for each of the seven schools studied. The 
program standards each of these accreditation 
agencies used for evaluating programs, pushed 
development of a "culture of evidence" with its basis in 
sampling student work across an entire program of study. 
Electronic portfolios were found to be valuable for this 
purpose. 

Each of the schools in this study found it necessary to start 
the ePortfolio design process by aligning their courses and 
programs of study with a wide range of standards. 
Curriculum Mapping was undertaken, with most schools 
addressing multiple sets of standards including: 1) 
national standards in all academic areas [INTASC, 
English-Language Arts, Math, Science, Technology, etc.]; 
2) state professional teaching standards; 3) NCATE or 
TEAC program evaluation standards; 3) U.S. Department 
of Education mandates and laws pertaining to special 
education and students who have special needs. To align 
courses and programs with various sets of standards, the 
schools used various technology resources as described 
in the cases below: 

UMD: A Standards/Syllabus Alignment Tool [SSAT] was 
designed to assist faculty of the College of Education with 
alignment of their course syllabi to a given set of 
standards. Faculty use a web-based system of forms to 
view the set of standards and indicate if their individual 
syllabi address those standards. 

Lesley: An online relational database is used to align 
courses in the Technology in Education program with 
goals and standards. Course assignments are aligned 
with the various standards, and the course professors 
review those relationships to ensure accuracy. The 
database is used to determine whether and how 
programs of study are aligned with national standards 
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and to respond to requests from our Regulatory Office for 
information about how the programs address standards 
in the various states where the research has been 
conducted. 

BHSU: Faculty completed a curriculum mapping project 
before implementing portfolios. Those curriculum maps, 
enabled to pull together a portfolio format that is based 
ontheINTASC standards. 

NLU: Colleges in the researcher's state (Chicago) are now 
working together to develop benchmark assessments 
and assignments that will address state professional 
teaching standards. 

Design Step 4 - Determination of the ePortfolio 
technologies to be used: 

Prior to making a decision about the specific electronic 
portfolio technologies to use, each school had to 
evaluate its technology infrastructure in order to identify 
how and in what ways that infrastructure could support the 
initiative. It was important to determine whether students 
would have adequate access to the technologies they 
would need to create, and maintain electronic portfolios. 
Schools need to determine whether their infrastructure 
was adequate for providing access to web or other online 
portfolio technologies. If not, or where funds were not 
available for building online portfolio infrastructures, 
electronic portfolios that used common software tools 
and CD's or USB drives for storage were chosen. Other 
common infrastructure concerns included adequate 
student access and support for the use of digital video. 
Many of the electronic portfolios used in the schools 
included a requirement that students provide video of 
teaching sessions and include these videos with 
reflections in their electronic portfolio. Part of the decision 
regarding the technologies to be used for portfolios 
included determining the nature of institutional, faculty 
and student access to student portfolio artifacts and 
assessments. Security concerns also included distribution 
of usernames and passwords, and decisions regarding 
who had access to aggregated assessment data. 
Allocation of financial resources, as well as commitments 
to future resource allocations by the institution, were 


considered vital to the success of electronic portfolio 
initiatives. All the schools found it necessary to provide 
financial resources for technology, faculty working on 
portfolio initiatives, and staff to assist with the electronic 
portfolio process. Financial resources were secured in a 
variety of ways, including grants for pilot studies and to 
help initiate electronic portfolio projects. 

UMD: The university provided financial and technical 
support for all phases of the electronic portfolio initiative 
including: the use of LiveText, the Standards-Syllabus 
Alignment Tool (SSAT), and the Performance Based 
Assessment tool (PBA) used online by faculty. 

UV: Grants from IBM were used to build an electronic 
portfolio system, which is made available to students 
state-wide. To ensure that the electronic portfolio process 
continues to evolve, there are now three different 
committees at the university working on the project. The 
Unit Assessment Committee oversees the organizational 
portfolio, a system designed to respond to accreditation 
data and self-improvement. The College's Technology 
Committee is developing a plan to include adequate 
support services to institutionalize electronic portfolios, 
and the Portfolio Committee is actively involved with state 
efforts to revise and improve the initial licensure process 
and its connection to student-constructed portfolios. 
Valdosta: The College of Education received two grants, 
PT3 (Preparing Tomorrow's Teachers to Use Technology 
Program) and GSTEP (Georgia Systemic Teacher 
Education Program). These have supported the 
development of electronic portfolios. 

WFU: WFU is a laptop school, and a totally wireless 
computing environment committed to pursuing the latest 
in electronic resources that support teaching and 
learning. The greatest challenge the department faces is 
providing enough digital video equipment to support the 
growing interest in video-editing technology to use with 
electronic portfolios. 

There was quite a bit of variability between the schools in 
their choice of electronic portfolio technologies. In 
several cases, electronic portfolio technologies changed 
over time as more uses for portfolio assessment were 
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defined within the school or institution. Excerpts from the 
case studies below highlight the variety of technologies 
chosen: 

UMD: Currently several eportfolio systems are being used 
at the university: LiveText, iWebfolio, and Open Source 
Portfolio Initiative (OSPi]. 

NLU: Beginning in 2003, both undergraduate and 
graduate students across the College were required to 
purchase LiveText to be used in support of program and 
college assessment. 

Valdosta: LiveText is used because it can house the 
College of Education portfolio template for preservice 
teachers, and allows faculty to evaluate preservice 
teachers' portfolios online. 

WFU: The use of candidate developed websites began in 
2000. Portfolios include digital video components. 
Candidate growth is documented with video clips 
selected from footage captured during student-teaching 
and videotape of the candidate reflecting on the 
significance of the clips to his/her growth as a 
professional. 

UV: At present, portfolios are generated using generic 
webdesign programs (Composer, DreamWeaver, and 
Word). Recently, the State of Vermont has been 
participating in a Reinventing Education Grant, (RE3) with 
IBM, to build a comprehensive portfolio system for the 
state to facilitate initial and re-licensure approval for 
teachers. Eventually there will be a state required 
electronic portfolio which demonstrates that students 
meet state and national professional standards. Through 
this grant, candidates for licensure have now been given 
access to Teachers' Workplace to create and present 
their re-licensure portfolios. 

BHSU: Students create a web based portfolio (Front Page 
was used by the researchers at this point) but they do not 
to publish it, except on the campus intranet. Students 
save their portfolio information on a CD or put it on a USB 
drive so that they can take their portfolio with them when 
they complete graduations. If they transfer, they can also 
take their information with them. 

Lesley: Thus so far far in both the Technology in Education 


and the Arts in Education Programs, eportfolio 
technologies have been limited to the use of PowerPoint 
or another multimedia design tools with hyperlinks to 
course assignments. These ePortfolios are copied to a CD 
and mailed to course faculty, then stored as evidence of 
student achievement of standards. Some students in 
Technology in Education choose to publish portfolios on 
the web. Students are not as yet provided with web 
access by the university; this is left to the individual student 
to find for themselves. Often students have web 
publishing access via their Internet Service Provider. Trial 
studies using a variety of electronic portfolio services have 
been undertaken. Those examined in trials with students 
include: Taskstream, LiveText; and ChalkandWire. There 
has also been some exploration of FolioTek and 
TrueOutcomes, though no trials have yet been 
completed with students. Both of these are specifically 
designed with extensive data management and 
reporting tools for program evaluation purposes, though 
they also include eportfolio services. 

Design Step 5 - Descriptions of portfolio requirements 
Portfolio requirements were typically defined by faculty 
through the use of templates or through a set of written 
requirements and instructions. Templates were designed 
specifically to be used with the eportfolio technologies 
chosen by the school. Most schools have more than one 
template, as these are customized to fit the needs of the 
different programs and departments within the college or 
university. Templates and/or written instructions provide 
information about the role of students in the portfolio 
process, the types and nature of artifacts to be included 
and also typically described the role(s) of reflection. 
Several of the schools used "Directed" portfolios with 
required assignments and reflections as a way of ensuing 
that the portfolio demonstrated student achievement of 
standards. 

UV: The Teachers Workplace webportal for the teachers 
defines the data to be included in the portfolio and has a 
tool licensure candidates use for aligning artifacts to 
standards. It also provides models of portfolios that 
exemplify quality. 
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NLU: The eportfolio program relies upon the use of 8 
benchmark assignments to provide evidence that 
students meet standards. Students select some of the 
benchmark assignments to include in their portfolios. 
Students are provided with a portfolio template, listing 
each required course in the program sequence along 
with required fields for data entry. Data entry fields 
include: (1) the artifact, (2) an artifact description, (3) 
standards met, (4) rationale for how the artifact 
demonstrates that students meet the standards listed, (5) 
reflection on what the project meant to the student's 
professional growth, and (6) reflections on the course 
including how the course may have helped them to meet 
diverse student needs and how the course enabled them 
to design or implement professional development 
activities. 

Once decisions regarding the technologies to be used 
were made and portfolio requirements were defined, 
most schools found that they needed to provide support 
to help faculty and students to gain necessary technical 
expertise and master the portfolio process. Technical 
support was designed to help both faculty and students to 
understand the various purposes of the portfolio, the 
portfolio process, and how to use the technology. 

Design Step 6 - Assessment systems for portfolio artifacts 
In all cases examined, a portfolio assessment process 
needed to be put in place. Issues that were addressed 
included: determining who would assess not only the 
individual artifacts in a portfolio but the entire portfolio and 
how that assessment would be done: the assessment 
tools (rubrics, etc.) to be used: and designing 
assessments to insure that student work demonstrated 
achievement of standards. Providing faculty with the time 
necessary for summative portfolio evaluation and 
reimbursement for that time were issues in some of the 
schools. 

UV: A (summative) portfolio rubric is provided by the state 
to promote reliable and uniform assessment. 

WFU: Formative assessment of candidate products 
occurs within the courses where the products are 
required. Summative assessment occurs in a variety of 


ways. The Department of Education faculty use external 
evaluations of candidate portfolios as an opportunity to 
receive feedback. The state required technology 
portfolios that are evaluated by a group of external 
evaluators that include local teachers and technology 
professionals from the school systems and the university. 
This process ensures that the program of study continues 
to address needs generated from the field. 

BHSU: A review team of three faculty members examine 
the portfolio using a rubric that has been designed to 
assess the portfolios in three major areas. These sections 
are weighted with #3 carrying the most weight: 1) Quality 
of the Personal Data Section: 2) Overall Organization and 
Appearance of the Portfolio: 3) Overall Quality of the 
Rationale Statement of the Self-Selected Artifacts 
(required artifacts and accompanying rationale 
statements have already been reviewed). 

NLU: Current portfolio evaluation process of NLV is very 
instructor dependent. Although the authors have a 
program-established rubric for evaluating portfolios, they 
find that some instructors waive selected requirements. It 
was pleasing to see a number of external visitors, who 
attended the final portfolio presentations. It has been, 
and continues to be, a challenge to determine the best 
way to collect feedback from these external sources. It is 
even a challenge to determine the most meaningful 
information to collect from this audience. 

Design Step 7 - Program evaluation systems for data 
gathering, aggregation, and analysis: 

Decisions had to be made by each of the schools 
regarding how data gathered through the use of 
electronic portfolios would be used for program 
evaluation. The tools and methods for data collection, 
aggregation and analysis had to be determined, and a 
process of analysis described and initiated. Technical 
training and support were also needed to ensure that the 
portfolio data collected could be aggregated and 
analyzed. 

NLU: The College Assessment Committee now requires 
annual Assessment Reports from each program as well as 
summary data which is collected by LiveText. The data 
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Design Step 1 

Defining the purposes of 
ePortfolios 

Purposes for portfolios as identified by the seven institutions of higher education included in this study: 
Support for Instruction and Student Assessment 

• Storage space for student work 

• Helping students connect theory and practice 

• Support for reflection on learning and teaching 

• Demonstrating growth in content understanding and pedagogical skills 

Support for Program Evaluation 

• Documentation of student progress toward meeting goals and standards 

• Documentation that students have met requirements for licensure 

• State and national accreditation and program approvals 

Design Step 2 

Defining the goals and standards 
to be assessed via ePortfolios. 

Multiple sets of standards were 
typically identified by the schools 
of education in this study. 

For Program evaluation purposes, ePortfolios needed to provide data that would allow schools to meet 
NCATE, TEAC, and/or the standards adopted by regional accreditation agencies. 
ePortfolio assessments needed to include evidence that teacher candidates met state professional 
teaching standards. 

EPortfolio assessments also needed to include evidence that teacher candidates could effectively 
teach state curriculum standards to their students (K-l 2 standards). 

Design Step 3 

Curriculum mapping - Matching 
goals and standards to the 
requirements of a program of study, 
by course, by licensure area, or for 
the whole program of study. 

All standard sets to assessed through an ePortfolio process needed to be analyzed to identify required 
competencies. "Standords crosswalks" or matrices to show relationships between sets of standards were 
developed. 

Course syllabi were examined to identify assignments, activities, and performance tasks demonstrating 
standards based / required competencies for each course in a program of study, and for an entire 
program of study. 

Many of the programs selected a set of required assignments and performance tasks to be included in 
the ePortfolio. These assignments were those that demonstrated important or required competencies. 

The result was a “Directed" ePortfolio in which students were not given total freedom to choose the 
artifacts to be included their ePortfolio (though in most cases students could add artifacts beyond those 
that were required). 

Design Step 4 

Determination of the ePortfolio 
technologies to be used. 

Most schools went through a process similar to the steps identified here. Evaluate the school's 

technology infrastructure. 

• Consider financing and budget for ePortfolio implementation and maintenance. 

• Preview ePortfolio systems. 

• Conduct trials using several selected ePortfolio systems. 

• Review technological requirements of preferred systems. 

• Consider technical implementation and support. 

• Gain consensus on one ePortfolio system based upon design, and implementation 
requirements. 

• Consider reporting systems reporting systems must provide access to important 
ePortfolio data for: students, faculty, academic programs, and for the institution as a 
whole. 

• Consider security issues for all ePortfolio information and data. 

Design Step 5 

Descriptions of portfolio 
requirements instructions providing 
information about the role of 
students in the portfolio process, as 
well as the types and nature of 
artifacts to be included. 

Facully defined ePortfolio requirements by creating written task descriptions or templates for selected 
course assignments and performance tasks. 

Facully defined additional ePortfolio design criteria - identifying extraneous features such as neatness, 
organization, graphics, and other required components 

Professional development for faculty in all aspects of the ePortfolio system became critical at this point in 
the implementation process. Faculty technology skills were an important consideration in the choice of 
technologies to be used and in the types of professional development necessary. However, professional 
development needed to be made available on a long-term and on ongoing basis. Students must also 
be provided with training on the ePortfolio system on a long-term and on ongoing basis 

Design Step 6 

Assessment systems for portfolio 
artifacts. 

Assessment methods needed to be defined for evaluation of student work and for determining whether 
students have met goals, standards and/or licensing requirements. 

Responsibility for assessing student ePortfolio work needed to be defined, particularly responsibility for fina 
ePortfolio evaluation. 

Design Step 7 

Program evaluation systems for 
data gathering, data aggregation, 
and analysis. 

How ePortfolio data would be used for program evaluation was determined by: 

• Determining what data from ePortfolios to gather 

• Defining a process for gathering data. 

• Determining the relationship of ePortfolio data to other data sets including: State 

teacher test performance. Student grades in their program of study. Student 

teacher or practicum evaluations. 

Decisions were made about who would aggregate and evaluate data. 

Facully and/or staff needed to be identified who would maintain, aggregate and analyze assessment 
data and keep assessment records necessary for program evaluation purposes. 

Design Step 8 

Using data to provide feedback for 
improving teaching, learning and 
program design. 

Data was used to provide feedback for improving teaching, learning and program design. 

Effective and efficient methods needed to be determined for using the ePortfolio system as tool for 
conducting program evaluation that results in program improvement. 

ePortfolios needed to be made efficient enough for gathering data to help answer questions such as: 

• "What do 1 need to do to help this student get to this standard?" 

• “What changes should 1 make in this course?" 

• "What changes need to be made in this program of study?" 


Table 1. Summary Table of Design Steps in the ePortfolio Process 
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shows how many students are "below", "meeting", and 
"exceeding" program expectations. 

UMD: A method was needed to aggregate the enormous 
amount of data required for NCATE and the state 
accreditation programs. The eportfolio system became a 
means to more easily aggregate, disaggregate and 
cross reference data across the college wide programs. 
The Performance Based Assessment System and 
Standards/Syllabus Alignment Tools, along with 
departmental and content unit assessments are also 
used. 

Valdosta: The LiveText portfolio system provides the 
College of Education with evidence that preservice 
teachers have met standards and a way to gather and 
evaluate data as evidence for accreditation. LiveText 
portfolio data is aggregated for university, state, and 
NCATE reports. Each academic program has aligned 
their program with the NCATE Standards, then created a 
LiveText portfolio template designed around those 
standards. Students submit artifacts as evidence of 
achieving each standard, and these are graded with a 
rubric. The data is aggregated in LiveText and presented in 
tables and graphs. LiveText is serving the important 
purposes of aggregation and evaluation of data. 

UV: An online tool is allowing faculty to collect assessment 
data from each of the 16 teacher preparation programs. 
These data sets include GPA, Praxis Scores, and a PADA 
(Professional Attitudes and Dispositions Assessment) which 
is completed by faculty as well as student. The university 
will be linking student ePortfolio artifacts and reflections 
into the data collection system so that they can 
demonstrate student performance outcomes. 

Design Step 8 - Using data to provide feedback for 
improving teaching, learning and program design 
This last step in an ePortfolio process is focused upon 
program improvement. There was general agreement 
among participants in this study that it is only when 
eportfolios provide information that faculty can use for 
improving the quality of courses and programs of study, 
that a feedback loop will be completed, ensuring high 
quality programs. It is only when such a feedback loop is in 


place, that all the national and state efforts of developing 
and implementing professional standards will produce 
highly effective teachers. Despite this understanding, 
data collection and aggregation was the final step 
described for most schools in this study. None of the case 
studies provided a clear, step-by-step description of how 
data analysis led to program evaluation or program 
improvement. 

UMD: The electronic portfolio component is considered 
one measure of student achievement of standards and is 
used with their Performance Based Assessment System for 
program evaluation. However, the report did not include 
a description of a process that made use of data sets to 
determine program effectiveness 
Valdosta: Portfolio assessment is used to provide 
evidence for accreditation as well as for internal 
evaluation of its programs. LiveText is able to provide 
aggregate data needed for both internal and external 
evaluations. However, there is no description of how that 
data is used to improve teaching and learning was 
included in the report. 

NLU: An assessment committee gathers data from 
LiveText on how many students are meeting standards. 
LiveText is able to provide aggregate the student 
achievement data needed for the state and NCATE 
accreditation programs. Again, however, there is no 
feedback system described that moved from data 
collection to the kinds of analysis of data that could be 
used for improving teaching and program design. 

Lesley: The ePortfolio system used in two programs does 
not provide for data aggregation or analysis, and 
therefore does not support any program evaluation 
process. However, the university is working toward 
adoption of a system that will be more effective for these 
purposes. 

Two schools are using evaluation of students' electronic 
portfolios rather than aggregated data to improve their 
programs of study. 

WFU: A process for using electronic portfolio presentations 
has been identified: "The faculty use external evaluations 
of candidate portfolios as an opportunity to receive 
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feedback. The state required technology portfolios are 
evaluated by a group of external evaluators that include 
local teachers and technology professionals from the 
school systems and the university. This process ensures 
that the program of study continues to address needs 
generated from the field". 

NLU: External evaluators are brought in during student 
eportfolio presentations to provide feedback. However, 
there has not yet been an effective method identified for 
using the feedback that these external evaluators provide 
for program improvement. 

Conclusions 

Cross case analysis in seven schools of education shows 
the impetus behind adoption of electronic portfolio 
systems, to have the result of regulatory requirements 
substantially for program evaluation from accreditation 
agencies including state education departments, 
NCATE, and TEAC. Electronic portfolios serve program 
evaluation purposes by providing storage space for 
student work samples which are used to demonstrate that 
students have met state, national and/or professional 
standards, and/or requirements for professional licensure. 
Cross case analysis shows effective electronic portfolio 
systems to have at least eight important design steps that 
include: 1) clear definitions of the purposes that 
eportfolios will play within the college or university: 2) 
identification of the goals and standards to be assessed 
via eportfolio: 3) curriculum mapping to match goals and 
standards to the task requirements of each course in a 
program of study: 4) identification of the electronic 
portfolio technologies to be used; 5) descriptions of 
portfolio requirements. Many schools used a partially 
"directed" portfolio process in which certain assignments 
were required because these demonstrated student 
achievement of multiple standards; 6) assessment 
systems such as rubric sets for determining whether 
portfolio artifacts met standards; 7) data collection 
systems for gathering, aggregating, and analyzing data 
to be used for program evaluation purposes; 8) a process 
for using the data gathered to improve teaching, learning 
and program design. This last step, involving the use of 


eportfolio data for program improvement has not yet 
been clearly defined by any of the schools of education 
in this study. These design steps were not necessarily 
addressed in the order listed by all schools, or given the 
same degree of importance. However, all design steps 
were found to be important to eportfolio processes 
intended to be used for regulatory and accreditation 
purposes. 

Despite the fact that the impetus for adoption of 
electronic portfolios was identified as being (at least in 
part) for program evaluation purposes, these case studies 
provide very limited information about exactly how 
eportfolio data are used for improvement in teaching, 
learning, course and/or program design. The focus was 
instead based upon such issues as the technologies 
chosen for collection, the storage of artifacts, and the 
training for faculty and students in the use of these 
technologies. Thus, findings indicate significant 
weaknesses in the program evaluation components of 
eportfolio processes. Further, this study raises questions 
about how eportfolio data is used to understand 
specifically what strategies actually work for helping 
teacher candidates to meet professional teaching 
standards. Information gathered through portfolio 
assessment is not being used to investigate which 
teaching strategies are specifically associated with 
educating great teachers. Instead, data collection is 
most typically defined as the final component. 

Few of the schools in this study addressed issues of data 
accuracy. Only UMD has the capacity for triangulation of 
data, as they collect results via electronic portfolio and 
their PBA system. However, other schools in this study are 
not gathering data that would allow them to do any 
triangulation, nor did the reports suggest that they are 
looking at student achievement information from 
multiple perspectives. None of the schools reported that 
they were using eportfolio data reports supplemented 
with graphic images, charts, or graphs to help faculty and 
students to advance to the next level of professionalism 
and achievement. Professional development designed 
to help faculty master the skills of advanced data driven 
decision making was not mentioned in any of the reports. 
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Where technology systems are used for data aggregation 
and analysis, questions must be raised about whether 
findings can be analyzed regularly enough and in a 
manner that makes them useful to the ongoing effort of 
program improvement. Reeves (2004) suggests the 
development of "Data Walls" for this purpose. Rather than 
having one "data guru" responsible for doing the analysis 
of results, he suggests that all faculty members get 
involved and use an entire wall of a room to post data, 
charts and graphs. Further, he states that these should not 
be static displays, but should be updated regularly and 
provide feedback that focuses on the most successful 
teaching techniques. Reeves states that such 
techniques help faculty to focus on identifying and 
replicating best practice. 

Overall findings raise questions about the effectiveness of 
eportfolio processes for program evaluation and 
improvement. Only when electronic portfolios provide 
information that faculty can use for improving the qualify 
of courses and programs of study, will a feedback loop be 
complete which ensures that all the national and state 
efforts of developing and implementing professional 
standards will produce highly effective teachers. 
Questions need to be asked regarding whether portfolios 
are actually an effective and efficient means of 
conducting program evaluation for improvement 
purposes. Can portfolios, even in their electronic version, 
be made efficient enough for gathering data that they 
helpanswerfacultyquestionssuchas: "What do I need to 
do to help the student to get to this standard?" "What 
changes should I make in this course?" "What changes 
need to be made in this program of study?" 
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